Domesticating AI in medical diagnosis

We consider the anticipated adoption of Artificial Intelligence (AI) in medical diagnosis. We examine how seemingly compelling claims are tested as AI tools move into real-world settings and discuss how analysts can develop effective understandings in novel and rapidly changing settings.Four case studies highlight the challenges of utilising diagnostic AI tools at differing stages in their innovation journey. Two ‘upstream’ cases seeking to demonstrate the practical applicability of AI and two ‘downstream’ cases focusing on the roll out and scaling of more established applications.We observed an unfolding uncoordinated process of social learning capturing two key moments: i) experiments to create and establish the clinical potential of AI tools; and, ii) attempts to verify their dependability in clinical settings while extending their scale and scope. Health professionals critically appraise tool performance, relying on them selectively where their results can be demonstrably trusted, in a de facto model of responsible use. We note a shift from procuring stand-alone solutions to deploying suites of AI tools through platforms to facilitate adoption and reduce the costs of procurement, implementation and evaluation which impede the viability of stand-alone solutions.New conceptual frameworks and methodological strategies are needed to address the rapid evolution of AI tools as they move from research settings and are deployed in real-world care across multiple settings. We observe how, in this process of deployment, AI tools become ‘domesticated’. We propose longitudinal and multisite `biographical’ investigations of medical AI rather than snapshot studies of emerging technologies that fail to capture change and variation in performance across contexts

Creating and Applying an Evaluation Framework for the National Decision Support Programme in Scotland

Context: The Scottish Government recognises the importance of decision support to improve knowledge management in health and care settings as a strategic priority. To this end, they funded the 2015 National Decision Support Roadmap. This laid out a plan for procuring and building a Decision Support Platform delivering a range of small-scale demonstrators (including several mobile platforms for specific user groups e.g. polypharmacy and diabetes), and building clinician and policy engagement for further funding. Aims: We were commissioned to undertake a formative evaluation of the National Decision Support Programme to help facilitate the effective roll-out of systems included in the Roadmap more widely. Methods: We collected qualitative data through a series of in-depth interviews and observations of workshops demonstrating technological systems. Participants included policy makers and clinical leads involved in the National Decision Support Programme. As the Programme was in the early stages of strategy development and system implementation at the time of data collection, we focused on exploring expectations and drivers of Cambio (a pilot platform) being tested in primary care. This system delivers an open standards based algorithm editor and engine which is linked with bespoke decision support applications delivered as web and mobile products and integrated into primary care electronic health record systems. The web and mobile solutions linked to the Cambio algorithms platform were developed by Scottish partners (Tactuum and University of West of Scotland). Employing a flexible methodological approach tailored to changing circumstances and need offered important opportunities for realising true impact through ongoing formative feedback to policymakers and active engagement of key clinical stakeholders. Our work was informed by sociotechnical principles and a health information infrastructure perspective. Qualitative data were coded with the help of NVivo software and analysed through a combination of inductive and deductive approaches. Findings: We collected data through 30 interviews and eight non-participant ethnographic observations of early stakeholder engagement workshops. We developed and applied a theoretically-informed evaluation framework, which we refined throughout our analysis. Overall, we observed a strong sense of support from all stakeholders for Cambio as an exemplar of an open standards based, customisable decision support platform, and proposals to roll this model out across NHS Scotland. Strategic drivers included facilitating integration of care, preventative care, patient self-management, shared decision-making, patient engagement, and the availability of information. However, in order to achieve desired benefits, participants highlighted the need for strong national leadership, system usability (which was perceived to be negatively affected by alert fatigue and integration with existing systems), and ongoing monitoring of potential unintended consequences emerging from implementations (e.g. clinical workloads). Conclusions and implications: In order to address potential tensions between national leadership and local usability as well as unintended consequences, there is a need to have overall national ownership to support the implementation of the Roadmap, whilst the implementation of individual applications needs to be devolved. This could be achieved through allowing a degree of local customisation of systems and tailoring of alerts, ongoing system development with continuing stakeholder engagement including “hands-on” experience for clinicians, a limited number of pilots that are carefully evaluated to mitigate emerging risks early, and development of a nuanced benefits realisation framework that combines smaller and locally relevant measures determined by implementing sites with national progress measures

Driving digital health transformation in hospitals:a formative qualitative evaluation of the English Global Digital Exemplar programme

BACKGROUND: There is currently a strong drive internationally towards creating digitally advanced healthcare systems through coordinated efforts at a national level. The English Global Digital Exemplar (GDE) programme is a large-scale national health information technology change programme aiming to promote digitally-enabled transformation in secondary healthcare provider organisations by supporting relatively digitally mature provider organisations to become international centres of excellence. AIM: To qualitatively evaluate the impact of the GDE programme in promoting digital transformation in provider organisations that took part in the programme. METHODS: We conducted a series of in-depth case studies in 12 purposively selected provider organisations and a further 24 wider case studies of the remaining organisations participating in the GDE programme. Data collected included 628 interviews, non-participant observations of 190 meetings and workshops and analysis of 9 documents. We used thematic analysis aided by NVivo software and drew on sociotechnical theory to analyse the data. RESULTS: We found the GDE programme accelerated digital transformation within participating provider organisations. This acceleration was triggered by: (1) dedicated funding and the associated requirement for matched internal funding, which in turn helped to prioritise digital transformation locally; (2) governance requirements put in place by the programme that helped strengthen existing local governance and project management structures and supported the emergence of a cadre of clinical health informatics leaders locally; and (3) reputational benefits associated with being recognised as a centre of digital excellence, which facilitated organisational buy-in for digital transformation and increased negotiating power with vendors. CONCLUSION: The GDE programme has been successful in accelerating digital transformation in participating provider organisations. Large-scale digital transformation programmes in healthcare can stimulate local progress through protected funding, putting in place governance structures and leveraging reputational benefits for participating provider organisations, around a coherent vision of transformation